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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
November 4, 2009 has been entered. 

Response to Amendment 

2. Claims 6-8, 11, 20, 33-34, 36, and 39 are canceled. 

Claims 1, 4-5, 9, 14, 18, 27, 30, 35, 37-38, and 40 are amended. 
Claims 1-5, 9-10, 12-19, 21-32, 35, 37-38, and 40-41 are pending. 

Response to Arguments 

3. Applicant's arguments with respect to the 35 U.S.C. 103(a) rejections of the 
pending claims have been considered but are moot in view of the new ground of 
rejections presented hereon. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-5, 9, 12-13, 17, 27-31, 35, 37, and 41 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Nakamura et al. (Pub. No. US 2004/0201740, 
filed on March 15, 2002; hereinafter Nakamura ) in view of Zarchan (Pat. No. US 
6,075,755, published on June 13, 2000), further in view of Andrews et al. (Pub. No. US 
2003/0060979, filed on September 21, 2001; hereinafter Andrews ), and further in view 
of Brechner etal. {Pat. No. US 6,741,996, filed on April 18, 2001; hereinafter Brechner ). 

Regarding claim 1. Nakamura clearly shows and discloses a method ([Abstract]) 
comprising: 

obtaining user provided information in consequence to any user operation on a 
mobile terminal device (Image capture device, such as a digital camera or a digital 
video camera 23, captures original images, [0021]); 

obtaining context information associated with said user provided information, 
wherein said context information is related to at least one current condition of the mobile 
terminal device at the time of said any user operation and includes calendar information 
from an electronic calendar implemented at said mobile terminal device {Image 
metadata is essentially non-picture information that is stored along with picture 
information in a file and can include information such as the date and time the picture 
was taken, whether a flash was used, which camera model was used, camera settings 
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such as zoom and exposure, location information such as GPS- derived data, and audio 
annotations, [0025]); 

obtaining a first time information in accordance with said user provided 
information (In step S314 image metadata that is stored along with each image is 
extracted. Image metadata are preferably date events that identify a date and time for 
specific events, [0031]); 

obtaining one or more calendar entries includes in said calendar information, 
wherein each calendar entry comprises a second time information with a start time and 
an end time (Each calendar in turn contains one or more events, each of which are 
preferably identified by an event title. Examples of event titles include birthdays for 
specific family members like "Dad's birthday" or "Nicholas's Birthday, " vacation events 
that correspond to vacations at specific times of the year like "Summer break, " and 
"Spring break", [0030]); 

comparing said first time information with each of said second time information to 
identify matching calendar entries (date events stored along with image as metadata 
are compared with dates corresponding to events specified in the calendar. Where an 
image meta data corresponds to more than one event on the calendar, and more than 
one meta data are available for the image, the other meta data are utilized to determine 
a storage location for that image, [0033]) and deriving a membership grade value from 
each of said membership functions in accordance with said first period of time (where 
two events are scheduled on the calendar for the same date, and the calendar 
schedules events by date and time, the comparing step compares the time the image 
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was recorded (meta data), if available, against the time of each of the calendar events 
on that date, and determines a location for the image accordingly, [0034]); 

obtaining meta-information from each matching calendar entry of said one or 
more obtained calendar entries (As shown in step S318, an affirmative response to this 
inquiry results in the creation of an album whose title is the title of the calendar event 
corresponding to the match, [0036]). 

Zarchan discloses: 

the comparison of said first time information and said second time information is 
by assigning a membership function to each of said second time information wherein 
said membership function is a function in time, which rises from zero value at a 
predetermined moment in time before said start time of a respective calendar entry, and 
becomes zero value at a predefined moment in time after said end time of said 
respective calendar entry, wherein membership function defines an extended timeframe 
for each calendar entry (The appointment message scrolls across the watch screen 
after the medicine regimen is displayed. The appointment reminders are displayed a 
number of days before the appointment and on the day of the appointment. They are 
canceled after the day of the appointment, [Column 10, Lines 50-60]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Zarchan with the teaching of 
Nakamura for the purpose of monitoring an individual activity and alarming such 
individual of upcoming events ([Abstract] of Zarchan ). 

Andrews discloses: 
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assigning meta-information obtained from said context information and said 
membership grade value to said user provided information {the data collection table 
31 5C is a table organized as a series of columns and rows. Each row defines a record 
for a particular appointment. The columns include an appointment column 502A, an 
alarm lead time column 502B, a calendar event column 502C, a GPS departure location 
column 502D, a GPS arrival location 502E, an arrival time column 502F and a stress 
factor column 502G. Each entry under the appointment column 502A contains an 
appointment number by which the particular appointment for that record can be 
identified, [0042]), said membership grade value defining a measure which allows to 
estimate a reliability of a timely relatedness for retrieval {stress factor in Figure 5); and 

directing storage of said user provided information and said meta-information in a 
history storage in order to establish an information history functionality {Referring now to 
Figure 7, a summary view 31 5B of the history data is shown. In this view, the history 
data contained in the history data structure 315 has been sorted into two columns 702 
and 704. Lead times and current GPS locations are contained in the history data 
structure among with other attributes, [0047]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Andrews with the teachings of 
Nakamura , as modified by Zarchan , for the purpose of notifying a user of an 
appointment by estimating a travel time for the user to travel from a current location of 
the wireless device to a appointment location ([Abstract] of Andrews ). 

Brechner discloses: 
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wherein said meta-information is employable for retrieval of said user provided 
information by matching request information of a retrieval request with said meta- 
information for selecting a user provided information assigned to said meta-information 
matched to said request information (the user could also specifically identify digital 
photographs relating to beach scenes by searching for metadata that include the word 
"beach" within the meta data associated with each of the files included within the 
collection hierarchy stored in the user's content database, [Column 1 1 , Lines 1 1-23]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Brechner with the teachings of 
Nakamura , as modified by Zarchan and Andrews , for the purpose of automating the 
indexing of media content in a simplified hierarchical storage scheme and the 
automated assignment of keywords and other metadata to the media content to 
facilitate its retrieval ([Column 1, Lines 5-10] of Brechner ). 

Regarding claim 2, Brechner further discloses said assigning comprises: 

extracting said meta-information from said context information (as the content is 
being imported into database created by the Clip Organizer, it simultaneously indexed 
the contents by adding keywords or metadata based upon contextual information, 
[Column 6, Lines 6-11]). 

Regarding claim 3, Brechner further discloses said obtaining of said user 
provided information comprises: 
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receiving user input information being generated by user operation of any input 
means of said mobile terminal device and/or 

receiving transaction information and/or communication information being 
received via any communication interface of said mobile terminal device {accessing 
those media files can be done with computing devices in distributed computing 
environments that include remote processing devices linked through a communication 
network, [Column 4, Lines 29-32]), 

wherein said user input information and/or transaction information and/or said 
communication information represent said user provided information {media files 
generated from digital cameras, web site image files, home video editing files, sound 
files, and other types of user media content are stored on a hard drive, [Column 1 , Lines 
14-18]). 

Regarding claim 4, Brechner further discloses said assigning of said meta- 
information to said user provided information comprises: 

extracting labeling information and/or indexing information from said context 
information (a user might store a digital photograph file "beachl jpg" that was taken with 
an OLYMPUS™ digital camera on the beach during a vacation to Hawaii in 1999 in a 
folder having a path such as "C:/My Documents/My Photo/Digital 
Camera/OlympusA/acations+HawaiH 999Beach1.jpg. " This path would be parsed by the 
Clip Organizer software to identify keywords for inclusion in the metadata for the file. 
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These keywords would include: "My Documents, " "My Photos, " "Digital Camera, " 
"Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47-57]); 

assigning said labeling information and/or indexing information to said user 
provided information (as the content is being imported into database created by the Clip 
Organizer, it simultaneously indexed the contents by adding keywords or metadata 
based upon contextual information, [Column 6, Lines 6-11]); and 

directing storage of said labeling information and/or indexing information and said 
user provided information (content is being imported into a database created by the Clip 
Organizer, metadata based on contextual information is also added to enable a user to 
find the personal content at a later time using a keyword search, [Column 6, Lines 6- 
11]); 

wherein said labeling information and/or indexing information is employed as said 
meta-information for establishing retrieval functionality of said user input history (The 
user could also specifically identify digital photographs relating to beach scenes by 
searching for metadata that include the word "beach" within the metadata associated 
with each of the files included within the collection hierarchy stored in the user's content 
database, [Column 11, Lines 11-23]). 

Regarding claim 5, Brechner further discloses said associating said context 
information with said user provided information comprises: 
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generating referencing information for at least a part of said user provided 
information (each collection in the database is associated a full path that indicates the 
location on the hard drive of the collection, [Column 9, Lines 53-55]); 

directing storage of said referencing information in a first storage area [the 
original media file with full path address would serve as a reference in order to import 
that file into the database created by Clip Organizer, [Column 9, Lines 48-59]); and 

directing storage of said part of said user provided information in a second 
storage area; wherein said referencing information comprises address information 
which allow to retrieve said part of user provided information from said second storage 
area (Provision of this full path is important, because it sets up an import and indexing 
component and a browse import component, [Column 9, Lines 55-65]). 

Regarding claim 9, Andrews further discloses partitioning said user provided 
information into a plurality information parts, each information part relating to a 
maximum pre-defined period of time, each information part being matched separately 
(the first lead time may define an alarm time six hours prior to the projected start time 
while a user is at home preparing for the appointment. A second lead time may be a 
lunchtime reminder with a lead time of two hours. A third lead time may be 15 minutes 
before the projected start time, [0031]). 

Regarding claim 12, Andrews further discloses each of said plurality of calendar 
entries being included in said calendar information is semantically structured and said 
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obtaining of meta-information from each matching calendar entry (Figure 5-6) 
comprises: 

obtaining of meta-information from each of said matching semantically structured 
calendar entry (Once the alarm time is reached, processing continues to step 908 
where the current GPS location and the appropriate calendar information (i.e., the user's 
current activity based on a calendar entry) are extracted, [0058]). 

Regarding claim 13, Brechner further discloses: 

obtaining said user provided information including user provided audio 
information and in parallel additional user provided information, said user provided 
audio information being recorded and stored (media files generated from digital 
cameras, web site image files, home video editing files, sound files, and other types of 
user media content are stored on a hard drive, [Column 1 , Lines 1 4-1 8]); 

obtaining said context information in parallel to said user provided audio 
information, said context information comprising user input information generated in 
consequence to user action against said mobile terminal device (a user might store a 
digital photograph file "beach 1 jpg" that was taken with an OLYMPUS™ digital camera 
on the beach during a vacation to Hawaii in 1999 in a folder having a path such as 
"C:/My Documents/My Photo/Digital 

Camera/OlympusA/acations+HawaiH 999Beach1.jpg. " This path would be parsed by the 
Clip Organizer software to identify keywords for inclusion in the metadata for the file. 
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These keywords would include: "My Documents, " "My Photos, " "Digital Camera, " 
"Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47-57]); and 

generating meta-information comprising information about said additional user 
provided information and said context information {as the content is being imported into 
database created by the Clip Organizer, it simultaneously indexed the contents by 
adding keywords or metadata based upon contextual information, [Column 6, Lines 6- 
11]). 

Regarding claim 17, Brechner further discloses said meta-information is 
displayed by assigning graphical elements to each information entry included in said 
meta-information and predicting said graphical elements illustrating the content of the 
meta-information and showing associations defined in said meta-information {Figure 
10). 

Regarding claim 27, Brechner further discloses a method for retrieving user 
provided information being organized, comprising: 

receiving a request for retrieving, said request comprising request information for 
instructing to retrieve certain user provided information {If the user wants to search for 
keywords in the metadata associated with the media files, the user will enter the likely 
keywords in a text block 354, [Column 15, Lines 37-44]); 

comparing said request information with said meta-information being assigned to 
said user provided information which is provided by said information history functionality 
{In this example, the user has entered the text "Vacation, " indicating that any media file 
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having that text string in its path and thus automatically included in the metadata for the 
file as a result of the automatic indexing of the media files will be located by the search, 
[Column 15, Lines 37-44]); 

retrieving said user provided information being assigned to said meta-information 
which matches said request information (any media file that has the searched word 
"Vacation" in its metadata will be located by the search, [Column 15, Lines 37-44]); 

generating a response comprising said retrieved user provided information 
(When the Search control is selected, a list of all media files meeting the desired criteria 
will be displayed to the user, [Colum 15, Lines 53-55]); and 

transmitting said response (the search results are displayed on the monitor, 
[Column 15, Lines 53-55] and Figure 1). 

Regarding claim 28, Brechner further discloses said retrieving of said user 
provided information comprises: 

retrieving referencing information being associated with said user provided 
information to be retrieved, said referencing information comprising address information 
which addresses said part of user provided information being stored in a second 
storage area (a user might store a digital photograph file "beachl jpg" that was taken 
with an OLYMPUS™ digital camera on the beach during a vacation to Hawaii in 1999 in 
a folder having a path such as "C:/My Documents/My Photo/Digital 
Camera/OlympusA/acations+Hawaii1999Beach1.jpg." This path would be parsed by the 
Clip Organizer software to identify keywords for inclusion in the metadata for the file. 
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These keywords would include: "My Documents, " "My Photos, " "Digital Camera, " 
"Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47-57]); and 

retrieving said part of user provided information from said second storage area 
{The user could also specifically identify digital photographs relating to beach scenes by 
searching for metadata that include the word "beach" within the metadata associated 
with each of the files included within the collection hierarchy stored in the user's content 
database, [Column 11, Lines 11-23]). 

Regarding claim 29, Andrews further discloses, wherein said retrieving of said 
user provided information comprises: 

evaluating said user provided information being retrieved on the basis of said 
membership grade values obtained from calendar entries, said membership grade value 
defining a measure which allows to estimate a reliability for retrieval {The particular 
appointment is identified in the appointment column 602A. The entries of the lead time 
column 602B are populated using the alarm lead time column 502B of the history data 
structure 315. Each entry of a projected lead time column 602C provides a lead time to 
be used for the next appointment. The projected lead time is calculated using a time 
difference value contained in the entries of a time difference column 602D, [0044]). 

Regarding claim 30, Brechner clearly shows and discloses a method for 
providing storage capacity for organizing user provided information being provided with 
meta-information {Abstract), comprising: 
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receiving a request to store at least a part of said user provided information 
(Figure 1 1 shows an interface to organize selected media clips on command. This 
interface could be displayed on hand-held devices, pocket personal computing devices, 
[Column 4, Lines 23-29]), wherein said request comprises said part of said user 
provided information and referencing information (Figure 11, #324 & #326); 

storing said part of said user provided information and said referencing 
information such that said part of said user provided information is retrievable in 
conjunction with said referencing information (a user might store a digital photograph file 
"beach 1 jpg" that was taken with an OLYMPUS™ digital camera on the beach during a 
vacation to Hawaii in 1999 in a folder having a path such as "C:/My Documents/My 
Photo/Digital Camera / Olympus / Vacations+Hawaii1999Beach1 Jpg. " This path would 
be parsed by the Clip Organizer software to identify keywords for inclusion in the 
metadata for the file. These keywords would include: "My Documents, " "My Photos, " 
"Digital Camera," "Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47- 
57]). 

receiving a request for retrieving at least said part of said user provided 
information wherein said request comprises referencing information (If the user wants to 
search for keywords in the metadata associated with the media files, the user will enter 
the likely keywords in a text block 354, [Column 15, Lines 37-44]); 

retrieving said part of said user provided information in accordance with said 
referencing information (In this example, the user has entered the text "Vacation," 
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indicating that any media file having that text string in its path and thus automatically 
included in the metadata for the file as a result of the automatic indexing of the media 
files will be located by the search, [Column 15, Lines 37-44]); and 

generating a response including said retrieved part of said user provided 
information (When the Search control is selected, a list of all media files meeting the 
desired criteria will be displayed to the user, [Colum 15, Lines 53-55]). 

Regarding claim 31, Brechner clearly shows and discloses a computer readable 
storage medium embedded with a computer program comprising programming code for 
carrying out the operations of claim 1 ([Column 4, Lines 12-17]). 

Regarding claim 35, Nakamura clearly shows and discloses an apparatus 
(Figure 1) comprising at least one processor and at least one memory storing computer 
program code, wherein the at least one memory and stored computer program code are 
configured to, with the at least one processor, cause the apparatus to at least: 

obtain user provided information (Image capture device, such as a digital camera 
or a digital video camera 23, captures original images, [0021]) and context information 
associated with said user provided information in consequence to any user operation on 
said apparatus, wherein said context information is related to at least one current 
condition of the apparatus at the time of said any user operation and includes calendar 
information from an electronic calendar implemented at said apparatus (Image 
metadata is essentially non-picture information that is stored along with picture 
information in a file and can include information such as the date and time the picture 
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was taken, whether a flash was used, which camera model was used, camera settings 
such as zoom and exposure, location information such as GPS- derived data, and audio 
annotations, [0025]); 

obtain a first time information in accordance with said user provided information 
{In step S314 image metadata that is stored along with each image is extracted. Image 
metadata are preferably date events that identify a date and time for specific events, 
[0031]); 

obtain one or more calendar entries includes in said calendar information, 
wherein each calendar entry comprises a second time information with a start time and 
an end time (Each calendar in turn contains one or more events, each of which are 
preferably identified by an event title. Examples of event titles include birthdays for 
specific family members like "Dad's birthday" or "Nicholas's Birthday, " vacation events 
that correspond to vacations at specific times of the year like "Summer break, " and 
"Spring break", [0030]); 

compare said first time information with each of said second time information to 
identify matching calendar entries by assigning a membership function to each of said 
second time information (date events stored along with image as metadata are 
compared with dates corresponding to events specified in the calendar. Where an 
image meta data corresponds to more than one event on the calendar, and more than 
one meta data are available for the image, the other meta data are utilized to determine 
a storage location for that image, [0033]) and deriving a membership grade value from 
each of said membership functions in accordance with said first period of time (where 
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two events are scheduled on the calendar for the same date, and the calendar 
schedules events by date and time, the comparing step compares the time the image 
was recorded (meta data), if available, against the time of each of the calendar events 
on that date, and determines a location for the image accordingly, [0034]); 

obtain meta-information from each matching calendar entry of said one or more 
obtained calendar entries (As shown in step S318, an affirmative response to this 
inquiry results in the creation of an album whose title is the title of the calendar event 
corresponding to the match, [0036]). 

Zarchan discloses: 

the comparison of said first time information and said second time information is 
by assigning a membership function to each of said second time information wherein 
said membership function is a function in time, which rises from zero value at a 
predetermined moment in time before said start time of a respective calendar entry, and 
becomes zero value at a predefined moment in time after said end time of said 
respective calendar entry, wherein membership function defines an extended timeframe 
for each calendar entry (The appointment message scrolls across the watch screen 
after the medicine regimen is displayed. The appointment reminders are displayed a 
number of days before the appointment and on the day of the appointment. They are 
canceled after the day of the appointment, [Column 10, Lines 50-60]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Zarchan with the teaching of 
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Nakamura for the purpose of monitoring an individual activity and alarming such 
individual of upcoming events ([Abstract] of Zarchan ). 
Andrews discloses: 

assigning meta-information obtained from said context information and said 
membership grade value to said user provided information (the data collection table 
31 5C is a table organized as a series of columns and rows. Each row defines a record 
for a particular appointment. The columns include an appointment column 502A, an 
alarm lead time column 502B, a calendar event column 502C, a GPS departure location 
column 502D, a GPS arrival location 502E, an arrival time column 502F and a stress 
factor column 502G. Each entry under the appointment column 502A contains an 
appointment number by which the particular appointment for that record can be 
identified, [0042]), said membership grade value defining a measure which allows to 
estimate a reliability of a timely relatedness for retrieval {stress factor in Figure 5); and 

storing said user provided information and said meta-information in a history 
storage in order to establish an information history functionality (Referring now to Figure 
7, a summary view 31 5B of the history data is shown. In this view, the history data 
contained in the history data structure 315 has been sorted into two columns 702 and 
704. Lead times and current GPS locations are contained in the history data structure 
among with other attributes, [0047]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Andrews with the teachings of 
Nakamura . as modified by Zarchan . for the purpose of notifying a user of an 
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appointment by estimating a travel time for the user to travel from a current location of 
the wireless device to a appointment location ([Abstract] of Andrews ). 
Brechner discloses: 

wherein said meta-information is employable for retrieval of said user provided 
information by matching request information of a retrieval request with said meta- 
information for selecting a user provided information assigned to said meta-information 
matched to said request information (the user could also specifically identify digital 
photographs relating to beach scenes by searching for metadata that include the word 
"beach" within the meta data associated with each of the files included within the 
collection hierarchy stored in the user's content database, [Column 1 1 , Lines 1 1-23]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Brechner with the teachings of 
Nakamura , as modified by Zarchan and Andrews , for the purpose of automating the 
indexing of media content in a simplified hierarchical storage scheme and the 
automated assignment of keywords and other metadata to the media content to 
facilitate its retrieval ([Column 1, Lines 5-10] of Brechner ). 

Regarding claim 37, Brechner further discloses: 

obtaining user provided audio information and additional user provided 
information (a user might store a digital photograph file "beachl jpg" that was taken with 
an OLYMPUS™ digital camera on the beach during a vacation to Hawaii in 1999 in a 
folder having a path such as "C:/My Documents/My Photo/Digital 
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Camera/OlympusA/acations+HawaM 999Beach1.jpg. " This path would be parsed by the 
Clip Organizer software to identify keywords for inclusion in the metadata for the file. 
These keywords would include: "My Documents, " "My Photos, " "Digital Camera, " 
"Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47-57]); 

recording said user provided audio information (before the sound/media files 
were stored on a computer's hard drive, it is inherent that the files had been 
recorded/downloaded/transferred from other sources, [Column 1, Lines 13-16]); 

directing storage of said user provided audio information (hard drive, [Column 1, 
Lines 13-16]); 

obtaining context information in parallel to said user provided audio information, 
said context information comprising user input information generated in consequence 
on a user action against said apparatus (a user might store a digital photograph file 
"beachl jpg" that was taken with an OLYMPUS™ digital camera on the beach during a 
vacation to Hawaii in 1999 in a folder having a path such as "C:/My Documents/My 
Photo/Digital Camera/OlympusA/acations+HawaiH 999Beach1.jpg." This path would be 
parsed by the Clip Organizer software to identify keywords for inclusion in the metadata 
for the file. These keywords would include: "My Documents, " "My Photos, " "Digital 
Camera," "Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47-57]); 
and 

generating meta-information comprising information about said additional user 
provided information and said context information (as the content is being imported into 
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database created by the Clip Organizer, it simultaneously indexed the contents by 
adding keywords or metadata based upon contextual information, [Column 6, Lines 6- 
11]). 

Regarding claim 41 . Nakamura further discloses said membership grade value 
indicates said timely relatedness between said user provided information and one of 
said matching calendar entries (where two events are scheduled on the calendar for the 
same date, and the calendar schedules events by date and time, the comparing step 
compares the time the image was recorded (meta data), if available, against the time of 
each of the calendar events on that date, and determines a location for the image 
accordingly, [0034]) 

6. Claims 10, and 15-16 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nakamura et al. (Pub. No. US 2004/0201740, filed on March 15, 
2002; hereinafter Nakamura ) in view of Zarchan (Pat. No. US 6,075,755, published on 
June 13, 2000), further in view of Andrews etal. (Pub. No. US 2003/0060979, filed on 
September 21, 2001; hereinafter Andrews ), and further in view of Brechner et al. (Pat. 
No. US 6,741,996, filed on April 18, 2001; hereinafter Brechner ), and further in view of 
Gupta et al. (Pat. No. US 6,484,156, filed on September 15, 1999; hereinafter Gupta ). 

Regarding claim 10, Nakamura , as modified by Zarchan , Andrews and Brechner , 
does not disclose in case said first period of time exceeds said second period of time: 
sectioning said user provided information into at least two information sections, one of 
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said at least two information sections fitting with said second period of time, said one 
fitting information section being matched. 

However, Gupta discloses in case said first period of time exceeds said second 
period of time {Figure 10 shows a target file is being divided into multiple segments, 
each segment has a corresponding play time, [Column 8, Lines 10-23]): 

sectioning said user provided information into at least two information sections, 
one of said at least two information sections fitting with said second period of time, said 
one fitting information section being matched (Figure 10 shows if the search time from 
000:04:08.262 to 000:11:00. 14 matches a segment, that segment will be retrieved when 
the 'Play' button is pressed. See further Figure 4 for all searchable fields in an 
annotation entry). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Gupta with the teachings of 
Nakamura , as modified by Zarchan , Andrews and Brechner , for the purpose of 
providing easy access to different multimedia streams by using a group identifier to 
identify all of the multimedia streams and their corresponding annotations ([Column 2, 
Lines 34-46] of Gupta ). 

Regarding claim 15, Gupta further discloses a recording context information at 
least including: 

information and time information about said user provided audio information 
(Figure 4, see Time Range #184); 
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time information about said user input information Figure 4, see Creation Time 
#188); and 

information about said additional user provided information being associated with 
the user provided audio information (Figure 5 shows the annotation entry used to 
associate with the media). 

Regarding claim 16, Gupta further discloses said generating of said recording 
context information comprises: 

encoding said meta-information as a document being encoded in accordance 
with a markup language (communication between client 15 and server 10 is performed 
via HTTP, using commands encoded as Uniform Resource Locators (URLs) and data 
formatted as object linking and embedding (OLE) structured storage documents, or 
alternatively using Extensible Markup Language (XML), [Column 6, Lines 9-15]). 

7. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (Pub. No. US 2004/0201740, filed on March 15, 2002; hereinafter 
Nakamura ) in view of Zarchan (Pat. No. US 6,075,755, published on June 13, 2000), 
further in view of Andrews et al. {Pub. No. US 2003/0060979, filed on September 21, 
2001; hereinafter Andrews ) , and further in view of Brechner et al. (Pat. No. US 
6,741,996, filed on April 18, 2001; hereinafter Brechner ). and further in view of Asazu 
(Pub. No. 2001/0049691, published on December 6, 2001). 
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Regarding claim 14, Nakamura , as modified by Zarchan , Andrews and Brechner , 
further discloses said user input information comprises control signals related to said 
recording of said user provided audio information, further comprises: 

on receiving a keyword signal: initiating a recording of a user provided audio 
keyword information including keywords relating to said user provided audio information 
(the file suffix is added as a keyword in the metadata for the file in a block 180. The 
suffix is the media file extension, such as "wav" for audio files in the wave format, 
[Column 10, Lines 32-44] of Brechner ): 

Asazu then discloses: 

on receiving a start signal: initiating said recording and directing storage of said 
user provided audio information (The following processing is executed in response to 
the query #2. Firstly, the QT plug-in component 14 calls a function of start () to the 
recorder 25 to issue an instruction to start recording of media data. Subsequently, the 
recorder 25 notifies the logger 24 of the instruction to start recording of media data, 
[0068]); 

on receiving an attach signal: associating additional information with said user 
provided audio information (In this case, the recorder 25 calls a function of notify ( ) to 
the recording manager 26 whenever each frame is processed. The recording manager 
26 activates the DA plug-in component 15 registered by the system in advance per 
frame data. Each DA plug-in component 15 executes an analysis of frame data to 
create meta-data, [0068]); 
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on receiving a pause signal: pausing said recording and storing of said user 
provided audio information (The following processing is executed in response to the 
query #3. The QT plug-in component 14 calls a function of stop ( ) to the recorder 25 to 
issue an instruction to stop recording of media data. The recorder 25 stops recording of 
media data, while notifying the logger 25 that recording of media data is stopped, 
[0069]); 

on receiving a continue signal: resuming said recording and storing of said user 
provided audio information subsequently to said pausing of said recording and storing 
(the processing for acquiring the information relating to meta-data is interrupted for the 
duration of the above access until the end of the other transaction. Thereafter, the QT 
plug-in component 14 resumes the processing to acquire the information relating to the 
required meta-data, [0078]. It is also very well-known in the art that resuming a paused 
recording part would start at the pausing point like in a digital voice recorder); and 

on receiving a stop signal: stopping said recording and storing of said user 
provided audio information and generating said recording context information (The QT 
plug-in component 14 calls a function of commit ( ) to the recorder 25, resulting in 
completion of recording of media data. It is to be understood that calling a function of 
abort ( ) instead of the function of commit ( ) aborts all the media data and meta-data 
recorded or created in this session after calling of the function of start ( ), and the 
system is restored to its original state, [0069]). 
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It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Asazu with the teachings of 
Nakamura , as modified by Zarchan , Andrews and Brechner , for the purpose of 
providing media data management functions including replaying, readout, recording, 
deletion and synthesis of media data by the use of meta-data ([0013] and Abstract of 
Asazu ). 

8. Claims 18, 23, 32, and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable Andrews et al. (Pub. No. US 2003/0060979, filed on September 21, 2001; 
hereinafter Andrews ) in view of Maxwell et al. {Pat. No. US 4,882,765, published on 
November 21 , 1989; hereinafter Maxwell ), further in view of Tecu et al. {Pub. No. US 
2004/0034655, filed on July 17, 2002; hereinafter Tecu ). and further in view of Brechner 
et al. (Pat. No. US 6,741,996, filed on April 18, 2001; hereinafter Brechner ). 

Regarding claim 18, Andrews clearly shows and discloses a method ([Abstract]) 
comprising: 

obtaining user provided information in consequence to any user operation on a 
mobile terminal device (Image capture device, such as a digital camera or a digital 
video camera 23, captures original images, [0021]); 

obtaining context information associated with said user provided information, 
wherein said context information is related to at least one current condition of the mobile 
terminal device at the time of said any user operation (Image metadata is essentially 
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non-picture information that is stored along with picture information in a file and can 
include information such as the date and time the picture was taken, whether a flash 
was used, which camera model was used, camera settings such as zoom and 
exposure, location information such as GPS- derived data, and audio annotations, 
[0025]); and 

directing storage of said user provided information with coded meta-information 
in a history storage in order to establish an information history functionality {Referring 
now to Figure 7, a summary view 31 5B of the history data is shown. In this view, the 
history data contained in the history data structure 315 has been sorted into two 
columns 702 and 704. Lead times and current GPS locations are contained in the 
history data structure among with other attributes, [0047]). 

Maxwell discloses: 

providing code basis representing a plurality of coding symbols, said code basis 
comprising a pre-defined number of pre-defined frequencies, wherein a plurality of 
coding symbols represents a character and symbol code table employable for coding 
said meta-information (Base station 10, mobile station 20 and repeater station 18 all 
receive and transmit on the same frequency F. In this arrangement each message is 
provided with a coding, say, in a header to he message indicating the number of times 
the message has been repeated, [Column 3, Lines 53-57]), wherein said set of code 
basis is defined within a first frequency range, which is one frequency range of a 
plurality of frequency ranges forming a total frequency range being applicable to said 
user provided audio information (operate the systems as above described with suitable 
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modification utilizing two separate frequencies wherein the base station transmits on a 
first frequency F1 and receives on a second frequency F2 that is different from F1. The 
mobile stations in turn transmit on frequency F2 and receive on frequency F1, and the 
repeater receives at least on frequency F2 and transmits on frequency F2, [Column 4, 
Lines 41-48]); 

repeating said code basis within at least one further frequency range out of said 
plurality of frequency ranges (the relay will repeat that message and retransmit again on 
the second frequency, this time for the benefit of the base station. When the mobile is in 
range of the base the base will receive the message from both the mobile and the relay 
stations, but if out of range only the repeated message from the relay station will be 
received, [Column 1, Lines 27-42]). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Maxwell with the teachings of 
Andrews for the purpose of relaying messages from one station to another through a 
relay station ([Column 1, Lines 12-13] of Maxwell ). 

Tecu discloses: 

coding said meta-information in accordance with said code basis defined within 
said first frequency range and repeated within said at least one further frequency range 
to obtain redundancy {At step 116, encoder routine 26 designates metadata 40 to be 
encoded at each of the selected frequencies 62. For example, each type of metadata 40 
to be included in the particular data stream 32 may be encoded at each of a plurality of 
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designated frequencies 62, [0028]. Encoder routine 26 may encode metadata 40 by 
generating a bit pattern at one or more desired inaudible frequencies 62, [0022]); and 

combining said user provided audio information and said coded meta-information 
by embedding said coded meta-information into said user provided audio information 
(metadata 40 may be encoded at a frequency 62 of approximately 20 kHz or greater, 
thereby rendering the encoded metadata 40 inaudible to human hearing. If data stream 
32 is to be compressed, the encoded metadata 40 may be inserted into audio data 34 
either before or after compression, thereby providing additional functionality and 
versatility to system 10, [00\19]). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Tecu with the teachings of 
Andrews , as modified by Maxwell , for the purpose of encoding metadata at a plurality of 
predetermined intensity levels at a human-inaudible frequency and populating the audio 
data of the data stream with the encoded metadata using an encoder routine accessible 
by the processor ([0003] of Tecu ). 

Brechner discloses: 

wherein said meta-information is employable for retrieval of said user provided 
information by matching request information of a retrieval request with said meta- 
information for selecting a user provided information assigned to said meta-information 
matched to said request information (the user could also specifically identify digital 
photographs relating to beach scenes by searching for metadata that include the word 
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"beach" within the meta data associated with each of the files included within the 
collection hierarchy stored in the user's content database, [Column 1 1 , Lines 1 1-23]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Brechner with the teachings of 
Andrews , as modified by Maxwell and Tecu . for the purpose of automating the indexing 
of media content in a simplified hierarchical storage scheme and the automated 
assignment of keywords and other metadata to the media content to facilitate its 
retrieval ([Column 1, Lines 5-10] of Brechner ). 

Regarding claim 23, Tecu further discloses a method for extracting meta- 
information from an audio information having embedded said meta-information, said 
method comprising: 

providing a correlation basis comprising each frequency being included in a set 
of code bases representing a plurality of coding symbols {Figure 1, frequency data 60), 
said coding symbols being employed for coding said meta-information (relational data 
50 may be generated after decoding of metadata 40 by decoder routine 28, or relational 
data 50 may be generated upon encoding or insertion of metadata 40 into a particular 
data stream 32, [0021]); and 

applying said correlation basis onto said user provided audio information having 
embedded said meta-information to extract said meta-information (Processor 16 also 
generates relational data 50 corresponding to the encoded metadata 40 such that 
metadata 40 may be correlated to particular data streams 32, [0021]); 
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wherein said extracted meta-information being available for retrieval (search 
engine 20 may be used to quickly and efficiently locate a particular data stream 32 
using search parameters corresponding to metadata 4, [0021]). 

Regarding claim 32, Andrews clearly shows and discloses a computer readable 
storage medium embedded with a computer program comprising programming code for 
carrying out the operations of claim 18 (Figure 3). 

Regarding claim 38, Andrews clearly shows and discloses an apparatus (Figure 
3) comprising at least one processor and at least one memory storing computer 
program code, wherein the at least one memory and stored computer program code are 
configured to, with the at least one processor, cause the apparatus to at least: 

obtain user provided information (Image capture device, such as a digital camera 
or a digital video camera 23, captures original images, [0021]) and context information 
associated with said user provided information in consequence to any user operation on 
against said apparatus, wherein said context information is related to at least one 
current condition of the apparatus at the time of said any user operation, wherein said 
user provided information includes user provided audio information (Image metadata is 
essentially non-picture information that is stored along with picture information in a file 
and can include information such as the date and time the picture was taken, whether a 
flash was used, which camera model was used, camera settings such as zoom and 
exposure, location information such as GPS- derived data, and audio annotations, 
[0025]); and 
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direct storage of said user provided information with coded meta-information in a 
history storage in order to establish an information history functionality (Referring now to 
Figure 7, a summary view 31 5B of the history data is shown. In this view, the history 
data contained in the history data structure 315 has been sorted into two columns 702 
and 704. Lead times and current GPS locations are contained in the history data 
structure among with other attributes, [0047]). 

Maxwell discloses: 

accessing a code basis representing a plurality of coding symbols, said code 
basis comprising a pre-defined number of pre-defined frequencies, wherein a plurality of 
coding symbols represents a character and symbol code table employable for coding 
said meta-information (Base station 10, mobile station 20 and repeater station 18 all 
receive and transmit on the same frequency F. In this arrangement each message is 
provided with a coding, say, in a header to he message indicating the number of times 
the message has been repeated, [Column 3, Lines 53-57]), wherein said set of code 
basis is defined within a first frequency range, which is one frequency range of a 
plurality of frequency ranges forming a total frequency range of said user provided audio 
information (operate the systems as above described with suitable modification utilizing 
two separate frequencies wherein the base station transmits on a first frequency F1 and 
receives on a second frequency F2 that is different from F1. The mobile stations in turn 
transmit on frequency F2 and receive on frequency F1, and the repeater receives at 
least on frequency F2 and transmits on frequency F2, [Column 4, Lines 41 -48]), wherein 
said code basis is repeated within at least one further frequency range out of said 
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plurality of frequency ranges (the relay will repeat that message and retransmit again on 
the second frequency, this time for the benefit of the base station. When the mobile is in 
range of the base the base will receive the message from both the mobile and the relay 
stations, but if out of range only the repeated message from the relay station will be 
received, [Column 1, Lines 27-42]). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Maxwell with the teachings of 
Andrews for the purpose of relaying messages from one station to another through a 
relay station ([Column 1, Lines 12-13] of Maxwell ). 

Tecu discloses: 

coding said meta-information in accordance with said code basis defined within 
said first frequency range and repeated within said at least one further frequency range 
to obtain redundancy (At step 116, encoder routine 26 designates metadata 40 to be 
encoded at each of the selected frequencies 62. For example, each type of metadata 40 
to be included in the particular data stream 32 may be encoded at each of a plurality of 
designated frequencies 62, [0028]. Encoder routine 26 may encode metadata 40 by 
generating a bit pattern at one or more desired inaudible frequencies 62, [0022]); and 

obtaining user provided audio information having embedded said meta- 
information into said user provided audio information (metadata 40 may be encoded at 
a frequency 62 of approximately 20 kHz or greater, thereby rendering the encoded 
metadata 40 inaudible to human hearing. If data stream 32 is to be compressed, the 
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encoded metadata 40 may be inserted into audio data 34 either before or after 
compression, thereby providing additional functionality and versatility to system 10, 
[00\19]). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Tecu with the teachings of 
Andrews , as modified by Maxwell , for the purpose of encoding metadata at a plurality of 
predetermined intensity levels at a human-inaudible frequency and populating the audio 
data of the data stream with the encoded metadata using an encoder routine accessible 
by the processor ([0003] of Tecu ). 

Brechner discloses: 

wherein said meta-information is employable for retrieval of said user provided 
information by matching request information of a retrieval request with said meta- 
information for selecting a user provided information assigned to said meta-information 
matched to said request information (the user could also specifically identify digital 
photographs relating to beach scenes by searching for metadata that include the word 
"beach" within the meta data associated with each of the files included within the 
collection hierarchy stored in the user's content database, [Column 1 1 , Lines 1 1-23]). 

It would have been obvious to an ordinary person skilled in the art at the time of 
the invention was made to incorporate the teachings of Brechner with the teachings of 
Andrews , as modified by Maxwell and Tecu , for the purpose of automating the indexing 
of media content in a simplified hierarchical storage scheme and the automated 
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assignment of keywords and other metadata to the media content to facilitate its 
retrieval ([Column 1, Lines 5-10] of Brechner). 

9. Claims 19, 21-22, 24, and 26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Andrews et al. (Pub. No. US 2003/0060979, filed on September 21, 
2001; hereinafter Andrews ) in view of Maxwell et al. (Pat. No. US 4,882,765, published 
on November 21, 1989; hereinafter Maxwell ), further in view of Tecu et al. {Pub. No. US 
2004/0034655, filed on July 17, 2002; hereinafter Tecu ). and further in view of Brechner 
et al. (Pat. No. US 6,741,996, filed on April 18, 2001; hereinafter Brechner ). and further 
in view of Tsuruoka (Pat. No. US 6, 192,056, published on February 20, 2001). 

Regarding claim 19, Tecu discloses said combining comprises: 

obtaining a modulation signal from said coded meta-information (intensity data 
70 having information associated with encoded metadata 40. For example, in the 
illustrated embodiment, intensity data 70 comprises signal amplitude or intensity levels 
72 used to encode metadata 40 such that various intensity levels 72 may be used to 
designate a particular bit pattern of information, [0017] of Tecu ). 

Andrews , as modified by Maxwell , Tecu , and Brechner , does not disclose the 
other limitations. 

However, Tsuruoka discloses: 

obtaining a modulated signal by combining said user provided audio information 
and said modulation signal in a frequency domain (a modulated signal using a 
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multiplicity of carriers whose frequency components are in an orthogonal relationship 
with one another, encodes data such as audio data or the like, and the encoded data 
are allocated to each carrier, thereby modulating each carrier, a digital signal in the 
frequency domain comprised of each modulated carrier is inverse fast Fourier 
transformed into a digital signal in a time domain, [Column 1, Lines 23-31]); and 

obtaining said user provided audio information having embedded said meta- 
information by combining said modulated signal with said user provided audio 
information in a time domain (On its demodulating side, byA/D converting such an 
OFDM modulated signal and then applying the fast Fourier transform to the AID 
converted signal, the encoded data allocated to each carrier is obtained, [Column 1 , 
Lines 28-31]). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Tsuruoka with the teachings 
of Andrews , as modified by Maxwell , Tecu and Brechner for the purpose of 
demodulating a digital orthogonal frequency division multiplex modulated signal in which 
an information signal modulates a plurality of carriers whose frequency components are 
in an orthogonal relationship with one another using OFDM (Orthogonal Frequency 
Division Multiplex) modulation ([Column 1, Lines 14-22] of Tsuruoka ). 

Regarding claim 21, Tsuruoka further discloses said obtaining of said modulated 
signal comprises: 
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obtaining said modulated signal by multiplying said user provided audio 
information and said modulation signal in said frequency domain (a modulated signal 
using a multiplicity of carriers whose frequency components are in an orthogonal 
relationship with one another, encodes data such as audio data or the like, and the 
encoded data are allocated to each carrier, thereby modulating each carrier, a digital 
signal in the frequency domain comprised of each modulated carrier is inverse fast 
Fourier transformed into a digital signal in a time domain, [Column 1 , Lines 23-31]). 

Regarding claim 22, Tsuruoka further discloses: 

obtaining said user provided audio information having embedded said meta- 
information by adding said modulation signal and said user provided audio information 
in said time domain (On its demodulating side, by A/D converting such an OFDM 
modulated signal and then applying the fast Fourier transform to the A/D converted 
signal, the encoded data allocated to each carrier is obtained, [Column 1, Lines 28-31]). 

Regarding claim 24, Tecu discloses said applying comprises: 

obtaining a modulation signal from said correlation basis (intensity data 70 
having information associated with encoded metadata 40. For example, in the illustrated 
embodiment, intensity data 70 comprises signal amplitude or intensity levels 72 used to 
encode metadata 40 such that various intensity levels 72 may be used to designate a 
particular bit pattern of information, [0017] of Tecu ): 

extracting magnitude signal values from said correlation signal (intensity data 70 
comprises signal amplitude or intensity levels 72 used to encode metadata 40 such that 
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various intensity levels 72 may be used to designate a particular bit pattern of 
information, [0017] of Tecu ), said magnitude signal values corresponding to each code 
basis of said set of code bases (encode metadata at a plurality of predetermined 
intensity levels at a human-inaudible frequency and populate the audio data of the data 
stream with the encoded metadata, [0003] of Tecu ): and 

evaluating said magnitude signal values to retrieve said meta-information from 
said user provided audio information (various intensity ranges 74 may also be used to 
designate a particular bit pattern of information. For example, a particular range of 
signal level strengths may be used to identify a bit designation of "1 " while another 
range of signal level strengths may be used to identify a bit designation of "0", [0017] of 
Tecu ). 

Tsuruoka discloses: 

obtaining a correlation signal from said user provided audio information by 
convoluting said modulation signal and said user provided audio information having 
embedded said meta-information in a frequency domain (a modulated signal using a 
multiplicity of carriers whose frequency components are in an orthogonal relationship 
with one another, encodes data such as audio data or the like, and the encoded data 
are allocated to each carrier, thereby modulating each carrier, a digital signal in the 
frequency domain comprised of each modulated carrier is inverse fast Fourier 
transformed into a digital signal in a time domain, [Column 1, Lines 23-31]). 
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Regarding claim 26, Andrews , as modified by Tecu , discloses said correlation 
basis being defined in a first frequency range (frequency data 60 in Figure 1, range is 
20kHz or greater to render inaudible to human hearing, [0019] of Tecu ) and said 
obtaining of said correlation signal comprises : 

providing a correlation basis comprising all frequencies of said set of code bases 
within a first frequency range of a plurality of frequency ranges {frequency data 60 in 
Figure 1, range is 20kHz or greater to render inaudible to human hearing, [0019] of 
Tecu ): 

mapping said correlation basis into each frequency range of a plurality of 
frequency ranges, said plurality of frequency ranges forming a total frequency range 
being applicable to said user provided audio information to obtain said modulation 
signal {Processor 16 also generates relational data 50 corresponding to the encoded 
metadata 40 such that metadata 40 may be correlated to particular data streams 32, 
[0021] of Tecu ): and 

Tsuruoka discloses: 

obtaining said correlation signal from said user provided audio information by 
convoluting said modulation signal and said user provided audio information in said 
frequency domain (a modulated signal using a multiplicity of carriers whose frequency 
components are in an orthogonal relationship with one another, encodes data such as 
audio data or the like, and the encoded data are allocated to each carrier, thereby 
modulating each carrier, a digital signal in the frequency domain comprised of each 
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modulated carrier is inverse fast Fourier transformed into a digital signal in a time 
domain, [Column 1, Lines 23-31]). 

10. Claim 25 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Andrews et al. {Pub. No. US 2003/0060979, filed on September 21, 2001; hereinafter 
Andrews ) in view of Maxwell et al. (Pat. No. US 4,882,765, published on November 21, 
1989; hereinafter Maxwell ), further in view of Tecu et al. (Pub. No. US 2004/0034655, 
filed on July 17, 2002; hereinafter Tecu ). and further in view of Brechner et al. {Pat. No. 
US 6,741,996, filed on April 18, 2001; hereinafter Brechner ). and further in view of 
Tsuruoka (Pat. No. US 6, 192,056, published on February 20, 2001), and further in view 
of Lew et al. (Pub. No. US 2002/0031240, filed on December 6, 2000; hereinafter 
Lew ). 

Regarding claim 25, Tecu further discloses extracting metadata using a 
magnitude value (various intensity ranges 74 may also be used to designate a particular 
bit pattern of information. For example, a particular range of signal level strengths may 
be used to identify a bit designation of "1" while another range of signal level strengths 
may be used to identify a bit designation of "0", [001 7] of Tecu ). 

Andrews , as modified by Maxwell , Tecu , Brechner , and Tsuruoka , does not 
disclose the first two limitations. 



However, Lew discloses: 
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obtaining magnitude ratio values of each pair of said magnitude signal values 
(For each of the M selected coefficients, x, the embedder computes a ratio of the 
magnitude of a selected coefficient relative to the magnitude of its neighbors (108). In 
particular, it is a ratio of the magnitude of the selected coefficient to the average 
magnitude of the surrounding neighbors, [0013]); 

normalizing said magnitude ratio values (The embedding and detecting 
operations apply to other media types, including audio media signals. In addition, the 
frequency domain coefficients may be selected and adjusted to reference values to 
detect other types of signal alteration, such as lossy compression, digital to analog and 
analog to digital conversion, downsampling and upsampling, etc, [0031]); and 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Lew with the teachings of 
Andrews , as modified by Maxwell , Tecu , Brechner , and Tsuruoka , for the purpose of 
detecting and analyzing alteration of a watermarked media signal that contains data 
information by examining signal peaks at selected frequency coefficients in the media 
signal ([0008] of Lew ). 

1 1 . Claim 40 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. {Pub. No. US 2004/0201740, filed on March 15, 2002; hereinafter 
Nakamura ) in view of Zarchan (Pat. No. US 6,075,755, published on June 13, 2000), 
further in view of Andrews et al. (Pub. No. US 2003/0060979, filed on September 21, 
2001; hereinafter Andrews ) , and further in view of Brechner et al. (Pat. No. US 
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6,741,996, filed on April 18, 2001; hereinafter Brechner ), and further in view of 
Kometani etal. (Pat. No. US 5,805,773, published on September 8, 1998; hereinafter 
Kometani ). 

Regarding claim 40, Nakamura , as modified by Zarchan , Andrews and Brechner , 
does not explicitly disclose the limitation of this instant claim. 

However, Kometani discloses deriving said membership grade value further 
comprises at least one out of averaging said membership function over said first period 
of time, determining a maximum of said membership function over said first period of 
time, and determining a minimum of said membership function over said first period of 
time (According to Formula (i), as shown in Figure 7, the membership functions 
(indicated by solid lines) of the conclusion part propositions M6, M7 and M8 of the rules 
R1, R2 and R3 are respectively weighted by the condition part membership grades 0.4, 
0.9 and 0.3 into the curves of the fuzzy-rule membership grades (indicated by broken 
lines), [Column 5, Lines 56-62]). 

It would have been obvious to a person with ordinary skills in the art at the time 
of the invention was made to incorporate the teachings of Kometani with the teachings 
of Nakamura , as modified by Zarchan , Andrews and Brechner , for the purpose of 
selecting one fuzzy rule group and at least one fuzzy set in conformity with a situation at 
that time and combines them and executing the reasoning for fact information 
([Abstract] of Kometani ). 
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Conclusion 

12. These following prior arts made of record and not relied upon are considered 
pertinent to Applicant's disclosure: 

Kohut et al. (Pub. No. US 2003/0187820) teaches media management system 
and process. 

Stubler et al. (Pub. No. US 2002/0188602) teaches method for associating 
semantic information with multiple images in an image database environment. 

Eval et al. (Pub. No. US 2004/0177096) teaches streaming media search 
system. 

The Examiner requests, in response to this Office action, support(s) must be 
shown for language added to any original claims on amendment and any new claims. 
That is, indicate support for newly added claim language by specifically pointing to 
page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the 
Examiner in prosecuting the application. 

When responding to this office action, Applicant is advised to clearly point out the 
patentable novelty which he or she thinks the claims present, in view of the state of the 
art disclosed by the references cited or the objections made. He or she must also show 
how the amendments avoid such references or objections See 37 CFR 1.111 (c). 
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1 3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Son T. Hoang whose telephone number is (571) 270- 
1752. The Examiner can normally be reached on Monday - Friday (7:30 AM - 4:00 PM). 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Christian Chace can be reached on (571) 272-4190. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/S.T.H/ 
Examiner, Art Unit 2165 
January 7, 2010 

/Neveen Abel-Jalil/ 

Supervisory Patent Examiner, Art Unit 2165 



